The touch screen of the Apple iPhone is recognized as a user interface breakthrough and, at least at the time of the device's introduction in 2007, was seen as somewhat radical in that the iPhone does not have any keypad or keyboard at all. The new multi-touch screen could detect multiple fingers touching the screen, was well integrated into the phone's operating system and featured in many applications. Regarding the absence of a keyboard, the market success of the iPhone demonstrates a form of market acceptance of requiring the user to enter phone numbers by touching a flat screen's displayed keypad or to enter text by typing on a displayed QWERTY keyboard. However, typing on an image of a QWERTY keyboard displayed on a flat surface is generally considered to be somewhat problematic when compared to using a real keyboard.
As discussed by Michael Kwan, Pros and Cons of Touchscreen Cell Phones, Mobile Magazine, August 2008, the absence of tactile feedback is a problem:
“There's just something to be said about hitting a physical button. For the life of me, I just can't type as fast on something like the Samsung Instinct or Apple iPhone as I can on something with a physical QWERTY keyboard like the LG enV2 or HTC Touch Pro. I also find that it's a lot easier dialing on a conventional numeric keypad than it is on a virtual keypad. Some handsets have tried to rectify this with haptic feedback, but it's just not the same.”
The author goes on to point out that ‘having to look to touch’ is a definite additional safety issue if a cell phone user is driving a motor vehicle and that a main attraction of a button on a ‘regular phone’ is that you know you pressed it.
More recently, Matt Braga in “How Haptic Feedback Brings Sensation to Touchscreens”, Tested, May 2010, speaks similarly to the touch screen typing problem then surveys today's attempts at a solution:
“It's for that reason that haptic feedback technology has become all the rage in recent years. Usually with the aid of a motor, haptic feedback aims to simulate the feeling of physical interaction while using a touch screen device. We've seen companies like RIM implement the technology with their Storm line of BlackBerries, while Motorola devices like the Droid have followed a similar approach. However, none of them feel quite right, so to speak. A vibration does little to simulate so-called ‘natural’ interaction, and that's a problem the latest haptic feedback technology is hoping to fix. The problem, however, is that the resulting vibrations are far from precise. Anything you touch seems to produce a similar, repetitive result, doing little to replicate the tactile sensation of a physical input.”
Toshiba's plan, announced May 2010, is to use Senseg's electrical haptic technology which modulates an electrical field to directly stimulate the touch sensors in the user's fingers. This approach is not only described in U.S. Patent Application 2009/0109007 assigned to Senseg a Finnish company founded in 2006, but also, in U.S. Patent Application 2010/0152794 assigned to Nokia, another Finnish company. The primary problem with this direct stimulation of the touch receptor approach is making the sensation that it produces recognizable to the user, for example like the feeling of a key on a keyboard.
Inventive efforts at Apple Inc. on haptic feedback for touch screens are evident in published U.S. Patent Applications 2009/0156818 and 2010/0156818. The latter application, Multi Touch with Multi Haptics, employs a phased array of haptic actuator/vibrators to create localized vibration feedback that appears to be an advance in comparison to the non-localized motor generated vibration feedback described above, but retains the problem of being a vibration. The former application, Multi-touch Display Localized Tactile Feedback, is closest to the present invention but focuses on how an application program would use such a technology with little contribution to actually creating the technology.
What is needed is a multiple touch screen with a dynamic tactile surface that can provide haptic feedback of sufficient fidelity that, for example, it can replicate the experience of using a physical keyboard. A transparent multi-touch sensing and tactile feedback screen assembly and an associated controller are desired that together are capable of providing both multi-touch input and local tactile response. Given the existing mutual capacitance technology for multiple touch detection and given the emerging capacitance-based technology of carbon nanotube actuators, what is needed is a means for combing the two technologies to provide a multi-touch screen with tactile feedback. Both of these technologies either use, or can use, a grid of x and y line electrodes in the screen assembly to provide localization. As a consequence, a specific need is for a touch and tactile screen controller system and method that allow the x and y line electrodes that are used for multi-touch sensing to also and simultaneously be used for localized actuator-based tactile feedback.